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T H E ED.D. PROGRAM AT THE UNIVERSITY OF ILLINOIS 
AT CHICAGO: USING CONTINUOUS IMPROVEMENT TO 
PROMOTE SCHOOL LEADERSHIP PREPARATION 


This article describes the process of replacing a modest Master s 
level school leader preparation program with an innovative Ed.D. pro¬ 
gram at the University of Illinois at Chicago (UIC). The new doctoral 
program is intensive, highly selective, intellectually rigorous, and field- 
based. The authors provide side-by-side comparisons of the difference of 
each inquiry cycle which identified both problems and opportunities that 
led to the changes in the old program. The results, the characteristics of 
the doctoral program are shown by the authors to have addressed weak¬ 
nesses of such leader preparation programs as identified in the research 
literature. Finally, the authors provide examples of specific data, such as 
student achievement data in schools led by UIC graduates, used in con¬ 
tinually improving the program. 


In 2001, a small group of faculty members in the Department of 
Educational Policy Studies at the University of Illinois at Chicago (UIC) 
challenged themselves by asking, “What would it take to consistently pre¬ 
pare urban school leaders to measurably improve student learning in high- 
need schools?” To answer this question and pursue the goal contained 
therein, we 1 embarked on a decade-long journey to improve the way we 
prepare school leaders at UIC. 

This article presents the story of the journey that has taken us from 
a modest Master’s-level school leader preparation program to an inten¬ 
sive, highly selective, intellectually rigorous, field-based doctoral-level 
program—the Ed.D. Program in Urban Educational Leadership. As we 
made this journey, we gradually moved away from radical, episodic, albeit 
collaborative approaches to program reform that have long been advocat¬ 
ed in the literature (Clark & Clark, 1997). Instead, we moved toward the 
regular use of data and cycles of inquiry to promote continuous improve¬ 
ment, an alternative approach more recently advocated for the improve¬ 
ment of leadership preparation (e.g, Bottoms, O’Neill, Fry, & Hill, 2003; 
Cheney & Davis, 2011; Orr, 2006). This move has proven crucial for us 
to engage in sustained program development and make notable progress 
toward the goal of consistently preparing urban school leaders to improve 
student learning in high-need schools. 

The story of the Ed.D. Program at UIC is of particular importance 
in light of public and scholarly attention to issues of school leadership 
preparation. This attention has spurred numerous critiques of leadership 
preparation (Barnett, 2004; Bottoms & O’Neill, 2001; Cheney & Davis, 
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2011; Cibulka, 2009; Finn & Broad, 2003; Fry, O’Neill, & Bottoms, 2006; 
Fless & Kelly, 2005; Levine, 2005; Murphy 2006; Murphy & Forsyth, 
1999). Critiques of university-based leadership preparation programs have 
been particularly negative (Bottoms & O’Neill, 2001; Cheney & Davis, 
2011; Cibulka, 2009; Fry et al., 2006; Fless & Kelly, 2005; Levine, 2005). 
Beyond general criticisms of program quality, specific concerns have fo¬ 
cused on lack of rigorous candidate selection processes; lack of program 
purpose, coherence, and rigor; antiquated program content; inappropriate 
pedagogical strategies; insufficient or poor quality field-based experienc¬ 
es; lack of quality university-district partnerships; and lack of quality pro¬ 
gram evaluation (Bottoms & O’Neill, 2001; Bredeson, 1996; Bridges & 
Flallinger, 1997; Fry et al., 2006; Fless & Kelly, 2007). 

From these criticisms has come a press for radically improving 
higher education school leader preparation programs. Complete program 
overhauls and new metrics for assessing program effectiveness have be¬ 
come prevalent in calls for reform (Bjork & Ginsberg, 1995; Bottoms & 
O’Neill, 2001; Bottoms et al., 2003; Cheney & Davis, 2011; Goldring & 
Schuermann, 2009; Norton, 2002). For example, Fry and her colleagues 
(2006) urged university departments of educational leadership 

... to awaken from their complacency, reject the status quo, and 
respond to appeals and criticisms from the field by identifying 
new content that addresses what principals need to know in order 
to do their jobs and by devising instructional processes that ensure 
principals master essential knowledge and skills, (p. 11) 

New perspectives on standards for assessing the quality and effec¬ 
tiveness of leadership preparation programs are also emerging and these 
standards are likely to drive reform. The Southern Regional Education 
Board (2002)articulates one such standard: that “every school has leader¬ 
ship that results in improved student performance” (p. 3). 

The bulk of existing literature on the reform of university-based 
school leadership preparation programs is wanting in several respects. Al¬ 
though the literature tracks steps that are being taken to reform school 
leadership preparation (Fry et al., 2006) and is replete with descriptions 
of innovative and exemplary leadership preparation programs, very little 
explicit attention has been paid to program impact and means to assess 
it (e.g., Cheney, Davis, Garrett, & Flolleran, 2010; Darling-Flammond, 
LaPointe, Meyerson, Orr & Cohen, 2007; Jean-Marie, Adams, & Gam, 
2010; Jackson & Kelley, 2002: Loss, 2009; Marsh & Dembo, 2009; Orr, 
2006; Storey & Hartwick, 2009; Toft-Everson, 2009). That is, there is vir¬ 
tually no empirical evidence that redesigned university programs, even 
those deemed innovative and exemplary, are making progress toward pre¬ 
paring school leaders to improve student learning. Also missing from ex¬ 
isting literature are descriptions of the work that is required to dramati¬ 
cally improve school leader preparation programs as well as examples of 
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robust student and program outcome data—data that are increasingly ex¬ 
pected for program evaluation and that are essential for informed program 
improvement. These gaps in the literature are concerning, given current 
expectations for leadership preparation programs to improve and to “eval¬ 
uate their impact on graduates and the schools their graduates lead” (Orr, 
2006 p. 498). 

This article begins to address these gaps in several ways, and by 
doing so aims to support others’ work to improve their programs of school 
leader preparation. First, this article provides an existence proof that a uni¬ 
versity-based leadership preparation program can reform itself and devel¬ 
op a record of preparing urban school leaders to improve student learning 
in high-need schools. Second, this article describes program design fea¬ 
tures that we have come to understand are necessary if such impact is to 
be achieved. Third, this article describes the use of data in program devel¬ 
opment and provides illustrations of program impact data that can be used 
to inform program improvement and evaluation. Equally important, this 
article illustrates the efficacy of cycles of inquiry and continuous improve¬ 
ment for sustained program development and effectiveness. 

The story of continuous improvement as a mechanism for promot¬ 
ing leadership preparation at UIC unfolds in three acts. In 2001, we en¬ 
gaged in a first formal cycle of inquiry that led to the initial planning and 
development of the program. Between 2002 and 2008, we launched the 
program and during this period of initial implementation, we began a pro¬ 
cess of continuous improvement for sustained program development. And 
from 2008 to the present, we enhanced our process of continuous improve¬ 
ment as we engaged in a second formal cycle of inquiry. This process al¬ 
lowed us to take stock of challenges and opportunities and led us to make 
several significant and important revisions to the program that we believe 
will further improve its effectiveness. Each of these three phases of our 
ongoing process of continuous improvement is detailed below. We fol¬ 
low these descriptions with a closer look at our data analysis approaches 
for program improvement and evaluation. We also discuss what we have 
learned regarding the challenges of a continuous improvement approach 
to ongoing program development. We end with concluding observations. 

Initial Planning and Development: The First Inquiry Cycle 

As the 1990s drew to a close, the Department of Educational Poli¬ 
cy Studies at UIC conducted a year-long program assessment that generat¬ 
ed two key conclusions similar to those later reached by Shulman, Golde, 
Conklin Bueschel, and Garabedian (2006) regarding education doctorates. 
First, our Ph.D. Program was not adequately achieving its primary objec¬ 
tive of preparing researchers, in large part because it was also trying to 
serve the professional development needs of practicing educators. We also 
found that neither the Ph.D. Program nor our M.Ed. Program was proving 
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effective in preparing school leaders capable of meaningful urban school 
improvement except as a rare exception. 

Motivated by these findings, we obtained a planning grant from the 
John D. and Catherine T. MacArthur Foundation in 2001 to begin a formal 
cycle of inquiry that led to the initial planning and development of the Ed.D. 
Program. This cycle was driven by the question that we asked ourselves as 
a faculty: “What would it take to consistently prepare urban school lead¬ 
ers to measurably improve student learning in high-need schools?” Finding 
little empirical research as guidance, a team of several Department faculty 
members and interested Ph.D. students turned to three sources of informa¬ 
tion: (a) an external needs assessment to identify demands and expectations 
for school leader preparation reform; (b) the literature on exemplary or in¬ 
novative programs of school leadership preparation; and (c) individual and 
focus group interviews of Chicago Public Schools (CPS) principals of low- 
income, racially isolated schools which had produced student learning out¬ 
comes that far exceeded expectations. We used the needs assessment to de¬ 
velop justification for a thorough reform of school leadership preparation in 
our department. We used our review of the literature and our interviews to 
gain new perspectives on how school leadership preparation might be re¬ 
formed. Particularly important was gaining successful school leaders’ per¬ 
spectives on transformational urban leadership practices and on how to help 
aspiring principals learn to employ them. 

This inquiry reinforced our belief that we would be unable to con¬ 
sistently produce new principals capable of transforming low-income ur¬ 
ban schools within the confines of an M.Ed. Program. The need for ex¬ 
tended academic work and intensive clinical experiences, as part of a 
continuum of pre-service preparation and early-career support and devel¬ 
opment, emerged as of paramount importance and could not be supported 
within our M.Ed. Program. Moreover, our M.Ed. Program attracted large 
numbers of students with very little teaching or leadership experience. We 
came to view such experience as an important prerequisite to successful 
principal preparation and effective leadership to transform low-perform¬ 
ing urban schools. With these ideas in mind, the department decided to 
close the M.Ed. Program and develop an Ed.D. Program as a new, more 
robust vehicle for initial school leader preparation and certification as well 
as for advanced professional leadership development. 2 

This initial inquiry cycle also pointed to a number of design ele¬ 
ments that would arguably improve on the design of the M.Ed. Program. 
We viewed these elements as conducive for making progress towards the 
Ed.D. Program’s primary objective of developing leaders to consistently im¬ 
prove student learning in high-need K-12 schools. In retrospect, we lacked a 
well-articulated theory of school leadership development to guide our think¬ 
ing. Table 1 compares the primary design elements of the M.Ed. Program 
with those of the Ed.D. Program as launched in 2002-2003. Notable are 
differences concerning the students we wished to work with; the selectiv- 
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ity of admissions; the intensity and duration of preparatory work; the strong 
emphasis on long-term field experiences and coaching; expanded focus on 
standards, practice-based competencies, and assessments; and the integra¬ 
tion of the theoretical with the practical, the academic with the clinical. 

Table 1 

UIC’s Original Master’s compared to Ed.D. at Launch in 2002 


UIC’s M.Ed. in 

educational administration pre-2003 

School system as consumer of 
candidates credentialed by UIC 


Non-selective admissions: most 
applicants to program accepted 


ISLLC standards as competency 
framework 


Single-strand program of 33 semester 
hours to credential wide variety of 
administrative roles, including 
department head, dean, athletic 
director; designed exclusively for 
pre-service training 

Revolving admissions spring, 
summer and fall, with 50-70 students 
at various stages of program 
completion; courses un-sequenced 
and taken full- or part-time as student 
chooses 

Limited clinical/residency experience 
provided; embedded in one course/ 
one semester and not requiring leave 
from current position; minimal 
on-site supervision 

No leadership coaching provided 


No consideration of academic and 
clinical integration and alignment 

Beyond course grades, no assessment 
of candidate perfonnance during 
program 


New Ed.D. program design features 
in 2002-03 startup year 

Close working partnership with public school 
system for program design, implementation, 
and assessment 

Highly selective admissions process requiring 
master’s degree, extensive documentation, and 
2-hour interview 

ISLLC standards plus 19 specific leadership 
competencies generated from interviews with 
urban school leaders 

Three-strand program structure of 88 to 92 
semester hours post Master’s degree designed 
for pre-service preparation and early career 
support and development of novice principals, 
advanced principals, and aspiring 
superintendents 

Cohort-based with admissions of 15-20 each 
fall; courses sequenced during year-long 
pre-service residency with enrollment in 3-4 
courses both residency semesters; variable 
sequencing during post-preparation, novice¬ 
leader development phase 

Extensive clinical experience through fully- 
paid, year-long principal residency (funded by 
CPS); close supervision by mentor principal 
and UIC coach; internship begins immediately 
at the start of program 

Three years of site-based leadership coaching 
(during year-long internship and 
post-internship for early-career development) 

Academic and clinical integration and 
alignment assumed rather than explicitly 
designed 

Beyond course grades, some basic assessment 
of candidate perfonnance throughout the 
program, with counsel-out consequences 

(continued) 
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Table 1 (continued) 


UIC’s M.Ed. in New Ed.D. program design features 

educational administration pre-2003 in 2002-03 startup year 


No capstone exam or demonstration Comprehensive exam, leadership portfolio, 
of leadership competence and thesis research focused on problems of 

leadership practice 


Staffing of roughly 2.0 FTE academic Staffing of roughly 3.0 academic FTE during 
faculty phase-out of Master’s; addition of 2.0 FTE 

clinical faculty (one internally funded and 
one externally funded) 


Launch and Early Implementation: 

Beginning Continuous Improvement 

The new Ed.D. Program for Urban Educational Leadership 
launched in 2002 with a first cohort of nearly 20 students. Although our in¬ 
ternal resources were stretched as we began the transition from the M.Ed. 
to the new Ed.D. Program, we received substantial support from CPS 
which funded full-year paid administrative residencies for students in the 
program. We also began to receive substantial support from philanthropic 
foundations primarily in Chicago that were interested in funding new ap¬ 
proaches to school leadership development. 3 

Between 2003 and 2005, we spent considerable time performing 
a broad range of “start-up” tasks. Program faculty developed curricular 
and instructional materials and began to develop a “conceptual framework 
for candidate and program evaluation.” This framework mapped program 
goals, evaluation strategies, and evaluation tools. In 2005, the department 
hired a new academic faculty member, expanding our capacity to imple¬ 
ment the program. 

Although the launch and early start up work had generally gone well, 
we began to experience several problems as implementation progressed and 
as our enrollment grew. 4 Beginning in 2005 and continuing through 2008, 
we engaged in both regular and more episodic periods of collaborative in¬ 
quiry to learn more about these problems and to make adjustments to ad¬ 
dress them. In the process, the faculty grew in its understanding of the im¬ 
portance of continuous improvement for sustained program development. 

For example, based on patterns of problems evidenced in stu¬ 
dent course taking, the internship, and comprehensive exams, we adjusted 
course sequences and improved horizontal and vertical course alignment. 
We amended course content and assessments, and we developed and sub¬ 
sequently revised the comprehensive examination and leadership portfolio 
requirements and procedures. We also adjusted coaching routines. Patterns 
of problems also offered insights regarding the selection and design of 
metrics and analytic processes for program improvement and evaluation. 
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Although our approach during this time frame was non-systemic 
and did not involve a formal cycle of inquiry, it produced improvement 
that otherwise could not have been achieved. Moreover, our commitment 
to program improvement and encouraging early evaluation results began 
to cultivate state and national attention. For example, an Illinois Board 
of Higher Education (2006) Commission on Principal Preparation named 
UIC’s Ed.D. Program as the state’s only “model” university-based princi¬ 
pal preparation program. By 2007, program improvement and evaluation 
data began to reveal success with various candidate placement indicators. 
For example, in job eligibility and placement we found: 

• Within their first three years in the program, 100% of UIC students 
passed CPS principal eligibility requirements compared to 46% of 
non-UIC candidates. 

• Of 34 UIC students who completed their internships in good standing, 
100% found jobs in CPS within three years. Almost two-thirds of these 
students were hired as principals, about one-quarter as assistant princi¬ 
pals, and the remainder as sub-district and district-level administrators. 5 

• Almost half of the UIC students receiving principalships were hired 
into high-need high schools. As a group, these students held nearly 
10% of all high school principalships in CPS. 

As we began to analyze candidate impact data, we also noted that 
our students were achieving modest success in improving academic achieve¬ 
ment in their schools. For example, in 2008 we found that taken collectively, 
elementary schools led by UlC-trained principals had improved 24.2 per¬ 
centage points in just four years on the meets/exceeds measures of compos¬ 
ite math and reading on the State ISAT achievement test, while CPS schools 
overall had improved 19.7% in that time. In posting these increases, UIC- 
led schools had moved from 3.5 points below the district average to surpass¬ 
ing the district average by 1 point: not earth-shaking, but trending positively 
against district scores that were themselves improving. Moreover, our im¬ 
provement work and results attracted the attention of The Broad Foundation. 
In fact, we were selected as the only university-based leadership preparation 
program to receive a major multi-year Broad grant. 

Despite these positive results and events, by the fall of 2008, our 
reviews of additional program evaluation data coupled with our nagging 
dissatisfaction with particular program issues, fueled unanimous agree¬ 
ment across the academic and clinical faculty for the need for a second 
more comprehensive cycle of inquiry to inform ongoing improvement. 
On the one hand, our data continued to show exceptionally strong rates of 
job placement after students’ internships. For example, all students in the 
program’s first seven cohorts who completed internships were hired into 
administrative posts by CPS within two years of those internships. In fact, 
ninety-five percent were hired immediately upon internship completion. 
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Moreover, two-thirds of our students obtained principalships within three 
years of their internships. On the other hand, our data also showed that 
15% of our students did not complete their internships. Further, the vast 
majority of the students (80%) who completed their internships showed 
signs of stalling during the post-residency phase of the program as they 
began their careers as full-time school or system-level leaders. 

Taking Stock and Making Larger Adjustments: 

The Second Cycle of Inquiry 

From 2009 to 2011 we engaged in a second formal cycle of in¬ 
quiry that was supported in part by external funding from Chicago Com¬ 
munity Trust, Fry Foundation, and McCormick Foundation. 6 In contrast to 
the first cycle that involved a small number of individuals, this cycle en¬ 
gaged all Ed.D. academic and clinical faculty, program staff, most Ed.D. 
students, and several faculty members from other departments within the 
College of Education. Work teams led by individual faculty members met 
for periods ranging from 6 to 18 months. 7 Equally important, this inquiry 
work expanded noticeably in scope as we collected a broad range of data. 
This process was a major undertaking that involved a substantial and sus¬ 
tained time commitment from academic and clinical program faculty. Our 
work was consistent with but arguably more multi-faceted than the pro¬ 
gram evaluation practices and recommendations emerging from the litera¬ 
ture (e.g., Clark & Clark, 1997; Educational Development Center, 2009; 
Fry et al., 2006; Story & Hartwick, 2009; University Council for Educa¬ 
tional Administration, 2011). 

This second cycle of inquiry focused our attention on strengths 
and weaknesses of existing program elements. We administered student 
surveys, conducted faculty interviews and student interviews and focus 
groups, and analyzed student course evaluations. We examined student 
course taking patterns, conducted a curriculum audit, and considered pass 
rates and patterns of problems in student comprehensive exams and in CPS 
principal eligibility assessments. 8 Importantly, we analyzed a growing as¬ 
sortment of data on candidate entry, progress, placement, and impact. 

We also collected and reviewed a broad range of information from 
outside sources to find points of comparison for further assessing the pro¬ 
gram’s strengths and weaknesses. For example, we drew from what we 
learned about exemplary leadership preparation program features through 
our ongoing participation in the Rainwater Leadership Alliance. 9 We re¬ 
viewed CPS principal eligibility competencies as well as ISLLC and NT- 
PBS Standards for School Leaders. We also drew on reviews of litera¬ 
ture on leadership preparation and literature describing exemplary school 
leadership programs. These reviews were complemented with our own in- 
depth investigations of several innovative programs at other universities. 
Finally, in anticipation of their inclusion in new state principal licensure 
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requirements, we reviewed literature on early childhood education leader¬ 
ship and content-area leadership. 

This inquiry cycle revealed a number of challenges and opportuni¬ 
ties for ongoing program development. Some of these challenges and op¬ 
portunities arose from external sources, such as changes in state program 
accreditation and principal certification requirements and in CPS job eli¬ 
gibility requirements. Others sprung from internal sources, such as grow¬ 
ing program size and complexity, limitations in faculty capacity, and un¬ 
derdeveloped administrative support. Some challenges had been identified 
earlier, such as difficulties students were having making steady progress 
during the post-residency portion of the program. Beginning in 2010, the 
program faculty began to respond to these challenges and opportunities 
by developing a number of changes to the program. These changes build 
upon the original 2003 program but in their scope and complexity can be 
seen as adding up to a program “redesign.” 

Table 2 recaps the Ed.D. design features at 2002, highlights select¬ 
ed problems and opportunities identified from the second formal cycle of 
inquiry, and presents Ed.D. design features that we began to phase in 2010. 

Table 2 

Ed.D. In 2002, Identified Problems and Opportunities, Ed.D in 2010 


Ed.D. program design 
features in 2002-03 
startup year 

State principal license: 
K-12 

State leadership prepa¬ 
ration policy allowed 
considerable flexibility 
with preparation program 
delivery 

Close working partnership 
with public school system 
for program design, imple¬ 
mentation, and assessment 


Highly selective admis¬ 
sions process requiring 
master’s degree, extensive 
documentation, and 2-hour 
interview 


Inquiry cycles identified 
following (selected) 
problems and 
opportunities 

State policy changes be¬ 
ing introduced related to 
leadership preparation and 
principal licensure 


Frequent changes in 
district policies create 
need for program changes 
but also opportunity to 
strengthen partnership 

Quality of candidate pool 
not up to our aspirations; 
suggests greater attention 
to pipeline development 
and improved selection 
process 


Program redesign 
components introduced or 
under consideration 

New PK-12 principal 
license beginning 2013; 
notable changes to the speci¬ 
ficity of clinical experiences 
and outcomes and with 
expectation for highly selec¬ 
tive candidate selection 

New Chicago leader¬ 
ship collaborative now 
underway; requiring new 
MOU and new (and likely 
strengthened) partnership 
structure with district 

New pipeline strategies 
enacted and additional 
strategies to be developed 
collaboratively with CPS 
through new leadership col¬ 
laborative; enhanced highly 
selective admissions process 

(continued) 
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Table 2 (continued) 


Ed.D. program design 
features in 2002-03 
startup year 

ISLLC Standards plus 19 
specific leadership com¬ 
petencies generated from 
interviews with urban 
school leaders 


Three-strand program 
structure of 88 to 92 se¬ 
mester hours post Master’s 
degree designed for pre¬ 
service preparation and 
early career support and 
development of novice 
principals, advanced prin¬ 
cipals, and aspiring super¬ 
intendents; little attention 
to identification and scaf¬ 
folding of programmatic 
content themes 

Cohort-based with admis¬ 
sions of 15-20 each fall; 
courses sequenced during 
year-long pre-service 
residency with enrollment 
in 3-4 courses both resi¬ 
dency semesters; variable 
sequencing during post¬ 
preparation, novice-leader 
development phase 


Extensive clinical experi¬ 
ence through fully-paid, 
year-long principal resi¬ 
dency (funded by CPS); 
close supervision by 
mentor principal and UIC 
coach; internship begins 
immediately at the start of 
program 


Inquiry cycles identified 
following (selected) 
problems and 
opportunities 

Insufficient program 
alignment to CPS principal 
competencies; opportunity 
to align with NTPBS arises 
with recent publication of 
their accomplished princi¬ 
pal standards 

Three-strand structure 
unwieldy for administer¬ 
ing and sustainability; 
complexity interferes with 
program content coherence 


Cohort effect diminishes 
as students pursue different 
career stages; academic 
challenge of program a 
factor in students stalling 
and not finishing program 
in timely manner 


Insufficient assessment of 
readiness and preparation 
for entry to internship 


Program redesign 
components introduced or 
under consideration 

CPS competencies, ISLLC 
standards, NBTS standards 
for school leaders and 
ISLLC standards integrated 
as indicators for pre-service 
and early-career school 
leader development 

Single-strand structure of 88 
credits post-Master’s degree 
proposed for all students 
with options/electives to 
accommodate specific career 
goals such as superintendent 
endorsement; programmatic 
content-based themes identi¬ 
fied and scaffolded across 
program experience 10 


Cohort-based with admis¬ 
sions of 15-20 each year 
moved up to spring provid¬ 
ing 18 rather than 12 months 
of pre-service training; 

Cohort experience improved 
through carefully sequenced 
courses; number of courses 
per semester in pre-service 
and development reduced 
(primarily by use of summer 
session) to accommodate 
school leadership demands 

Lully-paid, year-long 
principal residency (funded 
by CPS) begins after first 
full semester in program 
and based on assessment of 
candidate readiness; close 
supervision by mentor prin¬ 
cipal and UIC coach 

(continued) 
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Table 2 (continued) 


Inquiry cycles identified 

Ed.D. program design following (selected) Program redesign 

features in 2002-03 problems and components introduced or 

startup year opportunities under consideration 


Three years of site-based 
leadership coaching (dur¬ 
ing internship and post¬ 
internship for early-career 
development) 


Academic and clinical 
integration and align¬ 
ment assumed rather than 
explicitly designed 


Beyond course grades, 
some assessment of 
candidate performance 
throughout the program, 
with counsel-out conse¬ 
quences 

Comprehensive exam, 
leadership portfolio, and 
thesis research focused 
on problems of leadership 
practice 

Staffing: Staffing of 
roughly 3.0 academic 
FTE during phase-out of 
Master’s, plus addition of 
2.0 FTE clinical faculty 
(1 internally funded and 1 
externally funded) 


Coaching experience var¬ 
ies widely, suggesting the 
need for development of 
more explicit protocols, in¬ 
struments, and assessments 
to systematize coaching 
experience 

Weak alignment and inte¬ 
gration between academic 
and clinical experience 


Management of assess¬ 
ment process weak due 
to limited administrative 
capacity of program 


Faculty and student dis¬ 
satisfaction with poor 
student completion rates 
on original dissertation 


Academic staff stretched 
thin due to multiple re¬ 
sponsibilities of teaching, 
research, and program 
revision in an era of state 
budget retrenchment 


Three years of site-based 
leadership coaching main¬ 
tained, increased design and 
utilization of protocols, in¬ 
struments, and assessments 
for use during experience 

Redesign of internship- 
related courses to align with 
coaching; Explicit attention 
provided to alignment and 
integration between aca¬ 
demic and clinical experi¬ 
ence for entire program 

This area continues to need 
attention due to issues of 
capacity; recent receipt of 
additional university funding 
is likely to support improve¬ 
ments 

Introduction of capstone 
research project in place of 
more traditional dissertation; 
capstone model still under 
revision for recently-enter¬ 
ing cohorts 

Staffing of roughly 5.0 aca¬ 
demic FTE (one line added 
in 2005 plus parts of other 
faculty); 6.0 FTE clinical 
faculty (3 internally funded 
and 3 externally funded) 


As we begin this new chapter of the Ed.D. Program, our current 
enrollment stands at approximately 100 students. The demographic diver¬ 
sity of our student population has increased even as our admissions selec¬ 
tivity has increased. Almost 30 percent of our students are African Ameri¬ 
can, 15% are Latino, 42% are white, and 13 % are Asian American and 
other minority. Nearly 60% of students are female. The bulk of our stu¬ 
dents come to us from CPS and many are teacher leaders or assistant prin¬ 
cipals. Although the greatest numbers of students seek administrative cer¬ 
tification for the principalship, a number are experienced school leaders 
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seeking advanced professional studies and several are aspiring or early- 
career system-level leaders. 

We continue to post strong candidate placement data. For exam¬ 
ple, currently 96 students in eight UIC cohorts have completed the resi¬ 
dency year and achieved state certification. Of these 96, 65 became urban 
school principals within four years or less and nearly all of these posts 
were in high-needs schools. Of the remaining 31,30 have become system- 
level leaders or assistant principals. We also continue to chart progress 
with our student impact data. Our analysis of student impact data and an 
illustration of such analysis appears in the following section. 

A Closer Look at Data Analysis for 
Program Improvement and Evaluation 

A very important part of our work has been the ongoing develop¬ 
ment of a system of data collection and analysis for program improvement 
and evaluation. The objectives of this system have been consistent since the 
program was first developed. First, data and analysis should allow us to as¬ 
sess whether our program is achieving its overall goal of improving PK-12 
learning outcomes in schools led by our students. Second, they should allow 
us to assess whether program processes are being implemented with fidelity 
and quality on the way to achieving PK-12 learning outcomes. Third, they 
should enable us to improve program processes and outcomes through im¬ 
provements in program design and implementation. The pursuit of these ob¬ 
jectives has necessitated sustained work over the entire decade—work that 
has required a considerable commitment of faculty time and external funds 
to support additional “hands on” evaluation work of two program staff. 

The conceptual framework for candidate and program evaluation, 
developed shortly after the program launch, served as a foundation for 
conceptualizing, designing, enacting, and continuously improving a sys¬ 
tem of data for program improvement and evaluation. As currently enact¬ 
ed, this system assesses the full program trajectory which spans students’ 
admission to the program to their impact on the schools they serve. Ac¬ 
cordingly, we collect data at four key points: (a) student entry; (b) student 
program progress; (c) student job placement; and (d) student impact on 
their schools. We have learned that our data not only inform program im¬ 
provement and evaluation, but they also are an important source of infor¬ 
mation for students and their leadership work in schools. For example, im¬ 
pact data allow our students to consider whether their schools are closing 
achievement gaps over time and in comparison to multiple norms. 

More than 27 different data collection and analytic tools have 
been developed or selected from outside sources to assess our students and 
program. Not all of these tools have been employed and some which have 
been used have not been employed systematically. We have learned that 
a fully-developed system for improvement and evaluation requires sub- 
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stantial organizational capacity, and we have within the last several years 
been working to build our capacity for this work. Even though our work 
in this area continues to evolve, our current evaluation tools and strategy 
are beginning to inform national discussions on the evaluation of leader¬ 
ship preparation programs (Cheney et al., 2010; Cheney & Davis, 2011). 

During the past several years we have made the greatest progress 
collecting and analyzing data on our students’ impact on their schools. 
Specifically, we have engaged in more systematic annual and longitudinal 
analysis of several kinds of school-level data: (a) student attendance rates; 
(b) freshman academic on-track rates; (c) high school graduation rates; 
and (d) parent, student, and teacher ratings on various school culture and 
climate, instruction, and leadership indicators. We are also developing the 
internal capacity to use state databases that contain longitudinal standard¬ 
ized test score data on individual elementary and high school students for 
more robust analyses of learning in our students’ schools. By using state 
databases, we are beginning to perform “value-added” analyses and track 
longitudinal trajectories of student achievement by various subgroups 
of students (i.e. ethnicity and achievement quartiles) and in comparison 
to CPS and national trajectories. We are also able to compare the yearly 
learning gains of schools led by UIC students with district and state gains. 

At the end of the 2010-2011 school year, UlC-trained principals 
led 28 CPS elementary schools. These leaders had at least a full year’s re¬ 
sults on state achievement tests (ISAT), because they served in their re¬ 
spective schools for between one and seven years. Of these 28 schools, 
21 posted combined grade level 3-8 reading gains that exceeded CPS 
combined grade level 3-8 gains—some by a little, and some by a lot. 
Five schools of the 21 gained at least four months’ achievement, a useful 
benchmark that bodes well for genuinely closing achievement gaps over 
the course of an elementary school career because it indicates 1.4 years 
learning for one year’s schooling. Figure 1 is a data display that illustrates 
this analysis for the 21 UlC-led schools that outpaced CPS average gains. 
Moreover, our analysis shows that UlC-led elementary schools are twice 
as likely as CPS elementary schools to hit that benchmark of four months’ 
achievement gain. In mathematics, UlC-led schools have done even better, 
with eight of the 28 schools hitting the 1.4 year learning gain mark. More¬ 
over, in the composite analysis that combines reading and math scores, 10 
UlC-led schools of the 28 hit that high bar. 
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Figure 1. Elementary Principal Impact Data/UIC-Led School Compared to CPS Norm 
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Challenges of a Continuous Improvement Approach 
to Program Development 

Adopting cycles of inquiry and a continuous improvement ap¬ 
proach to program development has not been without challenges. Indeed, 
the challenges we have experienced are consistent with those generally 
experienced by organizations that move toward continuous improvement 
(Smylie, 2010). Consistent with the approach itself, these challenges have 
proven to be sources of learning about how to do the work of school lead¬ 
er preparation more effectively. Here, we describe briefly four challenges 
that have been particularly relevant to our journey. 

Challenge #1. Keeping Eyes on the Prize 

Of particular importance to continuous improvement are the artic¬ 
ulation, clarity, and consistency of mission and core values. Mission and 
core values, as well as the process of continuous improvement itself, are 
crucial “anchors” in organizations where regular changes in policies, pro¬ 
grams, and practices are to be expected (Smylie, 2010). However, main¬ 
taining the integrity and imperative of mission and core values is not easy. 
There is potential for goal displacement as organizations may train their 
focus on the process of change and indeed on change itself to the neglect 
of the outcomes to be achieved by change. There is also the potential for 
compromise as an organization engages with and responds to myriad de¬ 
mands and expectations of different external actors upon whom the orga¬ 
nization depends for resources and legitimacy. 

The mission and core values of the Ed.D. Program is well summa¬ 
rized in the question that has propelled our work since 2001: “What would 
it take to consistently prepare urban school leaders to measurably improve 
student learning in high need schools?” This question has served as a man¬ 
tra to the program. We are fortunate that the primary external actors upon 
whom the program depends now—CPS, the state, and philanthropic foun¬ 
dation - even our own college and campus—expect and demand from us 
much of what we expect and demand for ourselves. The greatest challenge 
to keeping our eyes on the prize is simply the pace and demands of the 
work itself and the risk that the “doing” and completion of work, or the de¬ 
velopment and institution of change, displaces attention on the outcomes 
we wish to achieve. That risk is mitigated but not eliminated by embed¬ 
ding the key elements of the question—“consistently prepare,” “to mea¬ 
surably improve student learning,” “in high need urban schools”—in the 
program’s design, content, and, importantly, evaluation system. 
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Challenge #2. Developing Complete Technology for Continuous Im¬ 
provement 

We have learned that data, data systems, and cycles of inquiry are 
very important to school leader program development. The challenge is to 
develop and manage a complete and coherent technology for continuous 
improvement that is anchored in mission and core values. The technology 
should be an integral, not ancillary or occasional, part of the program. As 
seen in our story, such a technology is also developmental and dynamic in 
nature. Moreover, the technology for continuous improvement is itself the 
subject of continuous improvement. That is, a truly complete technology 
is one that is turned on itself, is analyzed, and is improved on a continu¬ 
ous basis. It requires invention, continuous examination and development, 
and tolerance for ambiguity, risk, and occasional failure. However, devel¬ 
oping, managing, and improving a complete technology of continuous im¬ 
provement is especially challenging when there is little precedent for it in 
school leader preparation, and few resources to guide it. 

Of particular importance, our experience suggests that perhaps the 
greatest challenge to developing a complete technology is not in the de¬ 
velopment of data and data systems. Although there are substantial diffi¬ 
culties associated with developing data and data systems, these are largely 
technical matters. Instead, the greatest challenges come in developing the 
orientations, routines, and working relationships that have us continuous¬ 
ly looking at data and linking data to collaborative decision making about 
our students and program. 

Challenge #3. Developing Capacity for Continuous Improvement 

One of the most important lessons from the literature on continu¬ 
ous improvement is that it requires particular organizational capacities to be 
successful (Smylie, 2010). By capacities we mean organizational resources 
that support the work of continuous improvement. From our experience, we 
have learned the importance of cultivating the following capacities: 

• A culture of inquiry and continuous improvement that anchors firmly 
on mission, values, and process and makes change an expected ele¬ 
ment of life. 

• A culture of faculty collaboration, organizational citizenship, and dis¬ 
tributed leadership 

• The knowledge, skills, dispositions, processes, and management sys¬ 
tems to engage in cycles of inquiry, to analyze meaningful data, to use 
data effectively in planning and decision-making, and to use prod¬ 
ucts of that process—the new and improved aspects of school leader 
preparation produced 

• Program leadership that promotes mission, processes of continuous 
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improvement and processes to monitor, interact with, and manage the 
external environment 

• Organization of work, work roles and relationships, communications 
systems, and rewards systems to align with and provide support for 
continuous improvement work 

• Relational trust that provides both the “glue and lubricant” of work¬ 
ing relationships important for program personnel to deal with the 
uncertainty, risk, and occasional failure associated with continuous 
improvement (Tschannen-Moran, 2004) 

• Fiscal and physical resources, including time, which make important 
contributions to the cultivation of several capacities listed above 

Our challenge has been to foster sufficient capacity for a complex, 
intensive, evolving program. Indeed, as shown in our story, a number of 
the issues we have encountered in developing and implementing the Ed.D. 
Program can be traced to lack of fiscal resources including time. Fund¬ 
ing is scarce and highly competitive. Program personnel, particularly aca¬ 
demic faculty, must distribute their time and energy across a range of roles 
and responsibilities beyond those associated with the Ed.D. Program and 
its ongoing improvement. The Ed.D. Program has developed much of its 
“fiscal and time” capacities through external resources and unusually high 
levels of faculty commitment of time toward program improvement work. 
Our challenge going forward is to institutionalize the program by shifting 
its primary sources of financial support from unstable external sources to 
more sustainable sources within the university. The time commitments of 
individual faculty members for myriad improvement tasks must also be 
significantly adjusted in order to be sustainable. 

Challenge #4. Managing the Dilemma of Stability and Change 

This fourth challenge arises from the simultaneous need for the pro¬ 
gram to establish sufficient stability, regularity, and predictability to func¬ 
tion effectively and the need for the program to continuously adapt and im¬ 
prove. Too much of the former may compromise the ability of the program 
to respond effectively to problems and changing demands. Too much of the 
latter may create conditions resembling “permanent white water,” leading 
to problems of overload and fatigue that impose obstacles to achieving pro¬ 
gram objectives. Moreover, ongoing continuous improvement builds from 
periods of stability and “time off’ from what otherwise could become a re¬ 
lentless and unforgiving pace of change. This requires buffering people and 
their curriculum and instruction processes from frivolous demands so that 
human resources can be replenished (Smylie, 2010). 

Additional challenges of managing this dilemma at UIC come 
from the need to keep pace with continual changes in CPS’s job eligibil¬ 
ity requirements, as well as major reform in the state’s requirements for 


Vol. 43, No. 1/2, 2012, pp. 127-148 


143 


Cosner, Tozer, & Smylie 


school leader program accreditation and principal certification. The pace 
and magnitude of such change can create tension between our need to be 
responsive (our faculty’s inclination toward innovation) and the need to 
stick with particular program elements and practices long enough so that 
they can be both implemented well and fairly evaluated. The pace and 
magnitude of such change can strain program resources and tax program 
faculty, an important consideration. 

Concluding Observations 

Our experience, consistent with the program evaluation data col¬ 
lected thus far, suggest that institutions of higher education can indeed 
change and effectively enact innovative and robust programs of school 
leader preparation. The short history of the Ed.D. Program at UIC presents 
an existence proof of this possibility. Further, our story points to the prom¬ 
ise of cycles of inquiry, and the approach of continuous improvement for 
program development and long-term effectiveness also illustrates impor¬ 
tant details of this work and process. Although such processes have begun 
to be encouraged recently (Bottoms et al., 2003; Cheney & Davis, 2011; 
Orr, 2006), few illustrations exist in the scholarly literature on leadership 
preparation beyond our account. 

This story also demonstrates that a continuous improvement ap¬ 
proach to school leader program development is, to borrow the phrase 
from Elmore &McLaughlin (1988), “steady work.” It is difficult and com¬ 
plex work. Program development is not something to do and be “done.” It 
is ongoing. This approach requires strong commitment and sustained at¬ 
tention of all involved. The work is resource-demanding. It requires culti¬ 
vating and sustaining the right kinds of capacities in sufficient quantities 
and qualities. Importantly, it cannot be done well “on the cheap.” 

In conclusion, if cycles of inquiry and continuous improvement 
mark an important step in the maturation of the logic and practice of school 
leadership preparation and program development, then we in this field are 
almost certainly going to need to mature in our use of these processes. De¬ 
veloping the self-awareness, the sources of information, the routines, and 
the resources to engage in the process of continuous improvement of in¬ 
dividual programs and of the field will likely be critical areas of attention 
if the goal of consistently preparing school leaders to measurably improve 
student learning in high-need schools is to be realized. 

End Notes 

1 We authors use “we” in this story to connote the collective work of Department 
faculty. We do not mean to imply that every aspect of this story involved every 
faculty and staff member. We authors note where the work of individuals, small 
groups, and the faculty as a whole has particular meaning in the overall story. 


144 


Planning and Changing 


The Ed.D. Program at University of Illinois at Chicago 


1 The M.Ed. Program was later reopened as a program of introductory graduate 
work in educational policy studies and educator professional development, in¬ 
cluding teacher leadership. 

3 In addition to the John D. and Catherine T. MacArthur Foundation and The 
Broad Foundation, we have received generous external funding from the Chase 
Foundation, Chicago Community Trust, Chicago Public Education Fund, the 
Lloyd A. Fry Foundation, McCormick Foundation, McDougal Family Foun¬ 
dation, W. Clement and Jessie V. Stone Foundation, and the National Science 
Foundation. 

4 We admitted new cohorts of 15 to 17 students to the program each year, and in 
one year admitted two cohorts. 

5 Students hired at the district level were given substantial responsibility. Three 
of the 4 UIC students hired as sub-district administrators for designated geo¬ 
graphic areas within CPS collectively supervised the work of 32 principals. 

6 We have received roughly $5.5 million in external funds to support various 
features of our innovative program or for aspects of program development work. 
Another $9 million of funds have been provided by CPS to support residency 
salary and benefits. 

7 Teams were developed to examine several aspects of the program, including its 
mission, core values, and vision; curriculum and coursework; field experiences 
and coaching; the comprehensive examination and thesis research project. 

8 CPS introduced and subsequently revised on two occasions a process for gain¬ 
ing eligibility for CPS principal positions. At the present time interested candi¬ 
dates who hold or anticipate earning state principal licensure engage in a range 
of pencil-paper, performance, and role-play assessments where candidates are 
evaluated against a set of CPS Principal Competencies. Students typically begin 
this multi-step eligibility process during the final semester of their administrative 
internship. Given the dynamic nature of this process and identified competencies 
over the last five years, we have needed to be carefully attentive and responsive 
to changes with eligibility expectations. 

9 The Rainwater Leadership Alliance is a national coalition of school districts, 
universities, foundations, and nonprofit organizations dedicated to the improve¬ 
ment of school leadership. The Alliance exists to share information, provide 
exemplars, and promote and “scale” effective methods to develop and support 
PK-12 school leaders. 

10 Program curriculum now emphasizes three signature themes woven across 
multiple course experiences including: (a) cycles of inquiry and data use for im¬ 
provement, (b) leadership for social justice, and (c) leadership practice develop¬ 
ment and assessment. 
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